The present invention relates generally to data processing systems, and in particular, to temporary or permanent storage of bulk data distributions within networked data processing systems.
Present day data processing systems are often configured in large multi-user networks. Management of such networks may typically include the need to transfer bulk data to an endpoint system from a source system (or, simply, xe2x80x9ca sourcexe2x80x9d) and the collection of information, for example, error reports from a multiplicity of endpoints systems (or, simply, xe2x80x9cendpointsxe2x80x9d).
Such large data transfers may occur within a network, for example, to distribute software updates. The system administrator may need to allocate a specific period of time for the data transfer to more efficiently utilize network resources. This may typically occur when the communication load on the system is lowest, usually at night when most endpoint users are not working at their stations. The system administrator may load the bulk data and the corresponding transfer instructions onto the network system""s source, or server, in preparation for the transfer. At the predetermined time set by the administrator, the server will push the data while ensuring that the bulk data is successfully transferred to each of the desired endpoint locations. However, during the transfer a portion of the system server is dedicated to the data transfer and thus unavailable for other networking tasks. Moreover, as the number of endpoints which must be simultaneously serviced by the bulk data distribution increases, network bandwidth demands are concomitantly increased. This complicates scalability of the bulk-distribution systems.
Therefore, a need exists in the art for a bulk distribution mechanism that can transfer large amounts of data between network connected subsystems (or nodes) while maintaining scalability. Additionally, there is a need in such distribution mechanisms for methods and apparatus to distribute bulk data to a multiplicity of endpoints and to collect bulk data, including large log files, from the endpoints. These methods and apparatus may facilitate the bulk distribution of data by storing the data at intermediate nodes in the network, thereby reducing network bandwidth demand.
As described in the co-pending, commonly owned U.S. Patent Application entitled xe2x80x9cA Method and Apparatus for Distributing and Collecting Bulk Data Between Data Processing Systems,xe2x80x9d Ser. No. 438,437, incorporated herein by reference bulk data transfers are mediated by one or more repeaters between the server and endpoint. Each repeater includes a local repository which may be referred to as a xe2x80x9cdepotxe2x80x9d for the storage of data being transferred between servers and endpoints. Two uses of the depots are checkpoint restart functionality which allow an interrupted delivery to be automatically restarted from a predetermined point, and the ability to store commonly installed software distributions on nodes closer to their destinations. (Checkpoint restart is discussed in detail in the commonly-owned, co-pending U.S. Patent Application entitled xe2x80x9cApparatus for Restarting Interrupted Data Transfer and Method Therefor,xe2x80x9d Ser. No. 09/460,853, incorporated herein by reference.) For example, large software packages like Office 95(trademark) can be stored on local repeaters or gateway repeaters for fast and efficient distribution. A particular distribution can be stored in more than one repeater. For purposes of this application, the term depot is used interchangeably with repeater; however, it would be understood by an artisan of ordinary skill that, in general, repeaters incorporate functionality for depoting data in accordance with a portion of the depoting methodology of the present invention and discussed below in conjunction with FIG. 3, in addition to providing other functionality as discussed in the aforesaid U.S. Patent Application entitled xe2x80x9cA Method and Apparatus for Distributing and Collecting Bulk Data Between Data Processing Systems.xe2x80x9d